feat(replay): Add ReplaySnapshotObserver for snapshot testing#5386
feat(replay): Add ReplaySnapshotObserver for snapshot testing#5386runningcode wants to merge 11 commits into
Conversation
|
📲 Install BuildsAndroid
|
5b10cdd to
e7452f7
Compare
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 319f256 | 317.53 ms | 370.83 ms | 53.29 ms |
| ab8a72d | 316.24 ms | 356.38 ms | 40.14 ms |
| cf708bd | 434.73 ms | 502.96 ms | 68.22 ms |
| 91bb874 | 310.68 ms | 359.24 ms | 48.56 ms |
| f634d01 | 375.06 ms | 420.04 ms | 44.98 ms |
| 62b579c | 299.75 ms | 364.84 ms | 65.09 ms |
| 6b019b7 | 403.90 ms | 546.09 ms | 142.19 ms |
| d364ace | 384.53 ms | 453.51 ms | 68.98 ms |
| 5b66efd | 308.67 ms | 363.85 ms | 55.18 ms |
| ee35ac3 | 346.83 ms | 435.48 ms | 88.65 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 319f256 | 1.58 MiB | 2.19 MiB | 619.79 KiB |
| ab8a72d | 1.58 MiB | 2.12 MiB | 551.55 KiB |
| cf708bd | 1.58 MiB | 2.11 MiB | 539.71 KiB |
| 91bb874 | 1.58 MiB | 2.13 MiB | 559.07 KiB |
| f634d01 | 1.58 MiB | 2.10 MiB | 533.40 KiB |
| 62b579c | 0 B | 0 B | 0 B |
| 6b019b7 | 0 B | 0 B | 0 B |
| d364ace | 1.58 MiB | 2.11 MiB | 539.75 KiB |
| 5b66efd | 1.58 MiB | 2.13 MiB | 559.07 KiB |
| ee35ac3 | 1.58 MiB | 2.13 MiB | 558.77 KiB |
Previous results on branch: no/java-504-replay-before-store-frame-callback
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| a9ea107 | 311.04 ms | 361.61 ms | 50.57 ms |
| c906754 | 338.11 ms | 408.86 ms | 70.75 ms |
| cd28dc9 | 316.30 ms | 354.64 ms | 38.34 ms |
| 1cddad0 | 342.73 ms | 424.14 ms | 81.41 ms |
| 62bcea4 | 359.22 ms | 426.90 ms | 67.67 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| a9ea107 | 0 B | 0 B | 0 B |
| c906754 | 0 B | 0 B | 0 B |
| cd28dc9 | 0 B | 0 B | 0 B |
| 1cddad0 | 0 B | 0 B | 0 B |
| 62bcea4 | 0 B | 0 B | 0 B |
e7452f7 to
2aff4b0
Compare
| @Before | ||
| fun setup() { | ||
| // GH Actions emulators don't support capturing screenshots for replay | ||
| @Suppress("KotlinConstantConditions") |
There was a problem hiding this comment.
I copied this from ReplayTest but why are we even running these on emulators in gh actions?
markushi
left a comment
There was a problem hiding this comment.
LGTM! Let's discuss first if the API should be experimental for internal, but other than that no blockers.
| * intercepting frames for testing (e.g., screenshot comparison tests) or custom processing. The | ||
| * callback receives the frame after masking has been applied. | ||
| * | ||
| * <p>The frame bitmap is passed via a {@link Hint} using the key {@link |
0xadam-brown
left a comment
There was a problem hiding this comment.
Thanks for this @runningcode. A few comments.
Big picture, did we think about defining a new ReplayScreenshotObserver interface inside the replay module (akin to ScreenshotRecorderCallback) rather than routing through the (universally available) SentryReplayOptions?
An interface in the replay module would let us avoid the Hint indirection, and it'd avoid the tension of putting @ApiStatus.Internal on an *Options member.
Thoughts?
| * TypeCheckHint#REPLAY_FRAME_BITMAP}. On Android, retrieve it with: {@code hint.getAs( | ||
| * TypeCheckHint.REPLAY_FRAME_BITMAP, Bitmap.class)}. | ||
| * | ||
| * <p>The callback runs on a background thread (replay executor). Do not recycle the bitmap — it |
There was a problem hiding this comment.
l: We could beef this up to protect against misuse if we think it'd help:
* <p>**Bitmap lifecycle:** The bitmap is owned by the replay system. It should be deemed
* read-only and valid solely for the duration of the callback. Do not write to it or recycle it. Do
* not synchronize on it, store a reference to it, or access it after this method returns – copy
* the pixel data (e.g., compress to a file) within this method if you need it later.
*
* <p>The callback runs on a background thread (the replay executor)....or whatever makes sense.
There was a problem hiding this comment.
i've updated it to copy the bitmap. let me know if you think the new docs make sense along with that change to copy it
Add an experimental callback that fires right before a replay frame is
stored to disk. The callback receives the masked bitmap (via Hint),
timestamp, and current screen name. This enables snapshot testing of
replay masking without needing to decode stored video segments.
Includes a Kotlin extension for ergonomic usage:
options.sessionReplay.beforeStoreFrame { bitmap, ts, screen -> ... }
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…(JAVA-504) Add ReplaySnapshotTest that uses the beforeStoreFrame callback to capture masked replay frames during a Compose UI test. Frames are written to the Downloads/sauce_labs_custom_screenshots/ directory, which is the standard path Sauce Labs collects screenshots from. CI changes: - Add *.png to Sauce Labs artifact match patterns - Upload collected replay snapshots via sentry-cli build snapshots Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…VA-504) The Kotlin extension `beforeStoreFrame` comes from `sentry-android-replay` which may not resolve in the UI test module. Use the Java callback API directly instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…VA-504) GH Actions emulators don't support screenshot capture for replay, so the ReplaySnapshotTest needs the same assumeThat guard used by ReplayTest. Also adds a changelog entry for the beforeStoreFrame callback. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Markus Hintersteiner <markus.hintersteiner@sentry.io>
…r (JAVA-504) Move the frame observer API from the core sentry module to sentry-android-replay so it can use Bitmap directly instead of the Hint indirection. The new ReplaySnapshotObserver fun interface lives in the replay module and is set on ReplayIntegration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…in test (JAVA-504) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…AVA-504) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
375389b to
f2c0c49
Compare
|
|
||
| buildFeatures { buildConfig = true } | ||
|
|
||
| configurations.all { resolutionStrategy.force(libs.jetbrains.annotations.get()) } |
There was a problem hiding this comment.
I need to dig deeper in to why we need to add this to all the modules that add libs.jetbrains.annotations another day.
…VA-504) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…AVA-504) Move ReplaySnapshotTest to a conditional androidTestReplay source set so it's only compiled when APPLY_SENTRY_INTEGRATIONS is true. The test imports replay classes that aren't on the classpath otherwise. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…VA-504) Consumers of the observer API receive a copy of the bitmap instead of the replay system's shared instance. This eliminates race conditions and crashes when consumers store or use the bitmap asynchronously. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| observer.onSnapshotCaptured(copy, frameTimeStamp, screen) | ||
| } catch (e: Throwable) { | ||
| options.logger.log(ERROR, "Error in ReplaySnapshotObserver", e) | ||
| copy.recycle() |
There was a problem hiding this comment.
I would not have thought of calling copy.recycle() if we get an exception here. AI is smart.
| val file = File(snapshotsDir, "${name}_$frameTimestamp.png") | ||
| file.outputStream().use { out -> bitmap.compress(Bitmap.CompressFormat.PNG, 100, out) } | ||
| } | ||
| bitmap.recycle() |
There was a problem hiding this comment.
with the bitmap copy, we now have to make sure that we call bitmap.recycle() as should consumers of our API.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4a829fe. Configure here.
| captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp -> | ||
| val observer = snapshotObserver | ||
| if (observer != null) { | ||
| val copy = bitmap.copy(bitmap.config!!, false) |
There was a problem hiding this comment.
Unguarded NPE from bitmap.config!! skips frame storage
Medium Severity
The bitmap.config!! non-null assertion on line 316 is outside the try-catch block. Android's Bitmap.getConfig() can return null (e.g., for certain decoded image formats). If it does, a KotlinNullPointerException propagates out of the lambda, preventing addFrame on line 326 from ever executing. The ReplayExecutorService catches the exception so it won't crash, but the replay frame is silently lost. Setting a snapshotObserver can degrade replay quality even when the observer itself is well-behaved.
Reviewed by Cursor Bugbot for commit 4a829fe. Configure here.
| } | ||
|
|
||
| @Test | ||
| fun captureComposeReplayFrameSnapshots() { |
There was a problem hiding this comment.
just FYI, this is a pretty contrived test but it just makes sure we can capture a replay using the new snapshot observer api
| private val lifecycleLock = AutoClosableReentrantLock() | ||
| private val lifecycle = ReplayLifecycle() | ||
|
|
||
| @Volatile public var snapshotObserver: ReplaySnapshotObserver? = null |
There was a problem hiding this comment.
this awkward API is the consequence of moving the API to the sentry-andorid-replay api
| captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp -> | ||
| val observer = snapshotObserver | ||
| if (observer != null) { | ||
| val copy = bitmap.copy(bitmap.config!!, false) |
There was a problem hiding this comment.
we copy the bitmap now
| val observer = snapshotObserver | ||
| if (observer != null) { | ||
| val copy = bitmap.copy(bitmap.config!!, false) | ||
| if (copy != null) { |
There was a problem hiding this comment.
no point in calling the API if we have a null bitmap
| private val lifecycleLock = AutoClosableReentrantLock() | ||
| private val lifecycle = ReplayLifecycle() | ||
|
|
||
| @Volatile public var snapshotObserver: ReplaySnapshotObserver? = null |
There was a problem hiding this comment.
❗- Ah, I thought we were going the SentryReplayOptions route to avoid allowing folks to mutate configs after Sentry.init() returns (+ for enhanced visibility and better ergonomics)?
There was a problem hiding this comment.
Oh, then I misunderstood, this is the same API as in your branch which is what I thought we agreed to: a97de37#diff-cb8ccc45cef66aca485d205c9e99c17c114b6319ab9b7d52173b52ee72b047a8R126
We just need to make it public here instead of private in your branch because it needs to be usable from other packages i.e. other apps.
| private val lifecycleLock = AutoClosableReentrantLock() | ||
| private val lifecycle = ReplayLifecycle() | ||
|
|
||
| @Volatile public var snapshotObserver: ReplaySnapshotObserver? = null |
| captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp -> | ||
| val observer = snapshotObserver | ||
| if (observer != null) { | ||
| val copy = bitmap.copy(bitmap.config!!, false) |
There was a problem hiding this comment.
We should have our try block below enclose bitmap.copy(), esp given the !! 👍
There was a problem hiding this comment.
I think the only types of errors that can happen when copying the bitmap are memory errors and we shouldn't try to catch those.
As for the !! I don't see how bitmap.config can be null if it is a valid bitmap. Do you know how it could be null?
| } | ||
| } | ||
| addFrame(bitmap, frameTimeStamp, screen) | ||
| } |
There was a problem hiding this comment.
❗- It's not in the diff, so apologies for missing it the first time round, but I see there's an onScreenshotRecorded(File, Long) method as well as the Bitmap version above. I imagine we need to call the observer in both....
| * **Bitmap lifecycle:** The bitmap is a copy owned by the caller. You may store it or use it on | ||
| * another thread. Call [Bitmap.recycle] when you no longer need it to free native memory promptly. | ||
| * | ||
| * The callback runs on a background thread (the replay executor). |
There was a problem hiding this comment.
Worth emphasizing that folks should keep processing quick or hand off to another thread. (Otherwise they'll slow down the SDK's replay pipeline.)
| captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp -> | ||
| val observer = snapshotObserver | ||
| if (observer != null) { | ||
| val copy = bitmap.copy(bitmap.config!!, false) |
There was a problem hiding this comment.
Newbie question, but are we able to observe the performance hit of copying on our end?
Esp if we want to expand our use-cases beyond testing or debugging at some point, it'd be good to know whether we need to optimize.
There was a problem hiding this comment.
Memory-wise, each bitmap is roughly 900kb and we produce one frame per second.
| ### Features | ||
|
|
||
| - Add support to configure reporting historical ANRs via `AndroidManifest.xml` using the `io.sentry.anr.report-historical` attribute ([#5387](https://github.com/getsentry/sentry-java/pull/5387)) | ||
| - Session Replay: Add `ReplaySnapshotObserver` for observing captured replay frames ([#5386](https://github.com/getsentry/sentry-java/pull/5386)) |
There was a problem hiding this comment.
Worth noting that this is for use in snapshot tests and for debugging purposes and isn't currently optimized for production use.


Summary
ReplaySnapshotObserverfunctional interface that fires right before a replay frame is stored to disk, receiving the masked bitmap, timestamp, and screen namesnapshotObserveronReplayIntegrationso callers can set it after SDK initReplaySnapshotTestUI integration test that captures masked replay frames on Sauce LabsuseTestStorageServiceand*.pngartifact collection in the Sauce Labs configReplaySnapshotTestto a conditionalandroidTestReplaysource set so it compiles only whenAPPLY_SENTRY_INTEGRATIONS=trueRelates to JAVA-504
🤖 Generated with Claude Code